After taking a photograph, there are cases where users would like to change a composition of an object. In order to meet such a demand, the technique which changes an image after the image generation is known in the field of image processing.
In relation to this technique, National Patent Publication No. 2005-509985 discloses a method which defines an image area and modifies the composition of the image using interpolation in order to suppress a sense of discomfort. The technique described in the National Patent Publication No. 2005-509985 rectifies the gap generated by the composition modification using the pixel values of the surrounding pixels.
Moreover, a technique, which acquires information regarding the position, direction, and intensity of the light beams coming into the lens of a camera from an object by obtaining an image of an object shot from different viewpoints, is known. In relation to such a technique, the National Patent Publication No. 2008-515110 discloses a technique which acquires a plurality of images (light field images) in which an object is shot, and, from the plurality of images, the technique reconstructs the images of the object whose focal distances, depths of the field and the like are varied.
The information regarding the object included in the images shot from one viewpoint is limited to light beam arriving positions and light intensity at those positions (corresponding to the coordinates of each pixel and their pixel values, respectively). Thus, if an object composition is modified on a planar image, even if the object is modified using the technique described in the National Patent Publication No. 2005-509985, an interpolation not using the light beam information from an object is needed.
On the other hand, the National Patent Publication No. 2008-515110 describes a technique of acquiring information which shows a three-dimensional form of an object by photo shooting the object from a plurality of viewpoints, and a technique of acquiring light beam information and the like, from an object hidden and invisible from a certain viewpoint. However, the National Patent Publication No. 2008-515110 does not mention a method of generating the reconstructed image in which the composition of an object has been changed, and does not solve the above-mentioned problem.